Services and Pipelines

Many software components that run on the Rover are defined as a service. Though very fitting, this is not just a matter of naming: the service-oriented design we adhere to at ASE allows you and us to write highly optimized software for specific subtasks. A service can be written in any language and as long as you can encode communication between services to byte streams, you are good to go. Though to interface with the existing library of services that we offer you, it is important to understand how services work, how their runtime is controlled and how communication is arranged.

Elias Groot

Software Lead, Course Organizer

You might be surprised to hear about services in a closed-to-embedded system like the Rover. After all, micro-service based architectures are much more common in larger distributed applications that run on powerful compute in data-centers. Communication between two separate processes almost always increases latency compared to mere function calls, and additional orchestration is needed to make sure that services can find and talk with each other.

A lot of downsides at first sight, but don't stop reading there. For the same reason that big cloud providers like them, we at ASE are also fond of our services. Services isolate specific components which can then be implemented in a highly optimized manner (think C++ for your motor drivers, or using Python for your Computer Vision models), they bring resiliency (a failing service can be restarted without crashing other services) and allow for improved debugging (the communication between services can be transparently intercepted and logged). Besides, if your requirements change, you can just swap out service implementations as long as the communication remains unchanged. Oh, and did I talk about the beauty of unit testing yet?

The point is clear: having a service-oriented software architecture works well for us, and gives you as a developer great flexibility in how you set up your project. We'll walk through services at ASE step by step: first looking at the practical side (files and folders) and then gradually moving to collections of services that communicate to work together.

Defining a Service (service.yaml)

Let's start simple: a service in essence is nothing more than a program: a binary or script that is stored in one or more files. Every service should be captured in its own folder, with in the root of this folder a service.yaml definition - this is the heart and soul of a service.

For example, a folder structure might look like something like this for a service written in C:

/my-first-service
  ├── service.yaml
  ├── /src
        ├── main.h
        ├── main.c
  ...

You will read and write a lot of service.yamls throughout ASE. When using services from others (like the ones that we developed for you to use), it is always useful to read its service.yaml first. Its purpose is fourfold:

• Define the service identity: what service is this? What does it do? Who is the author and what version are we using?
• Understand the service input and output streams: how can other services read or write to this service?
• Understand how the service is built and run: how can the service be compiled/transpiled and executed?
• Tune the service options: what options are available? How can we modify them without re-compiling or even restarting the service?

Sounds abstract? Let's make it more concrete by going over an example. The following is a real service.yaml that defines our imaging service.

imaging/service.yaml

name: imaging
author: vu-ase
source: https://github.com/vu-ase/imaging
version: 1.0.1
description: Reads an image from a video source (v4l2), thresholds it and extracts track edge information

commands:
  build: make build
  run: ./bin/imaging -debug

inputs: [] # does not depend on other services

outputs:
  - path

configuration:
  - name: img-width
    type: number
    tunable: false
    value: 640
  - name: img-height
    type: number
    tunable: false
    value: 480
  - name: img-fps
    type: number
    value: 30

Even without further explanation, this should give you a fairly complete picture of what the imaging service does for you: it writes track edge data to an output socket identified by path and can be configured with different camera options like width, height and FPS.

Service Identity

imaging/service.yaml

name: imaging
author: vu-ase
source: https://github.com/vu-ase/imaging
version: 1.0.1
description: Reads an image from a video source (v4l2), thresholds it and extracts track edge information
...

The first five lines define the service identity. The name, author and version are especially important since, together, they make up for a services' Fully Qualified Name (FQN). Any service can be uniquely identified by its FQN, which in this case is vu-ase/imaging/v1.0.1. You will later see that the FQN is closely related to the folder structure in which services are saved on the debix.

By using versioning for the services you write, you can quickly A/B test and roll back to previous versions when necessary. Our software framework has first-class support for versions and through roverctl you can quickly switch between versions.

The name, author, source and version fields are required. Specifying the description is optional yet recommended.

Build and Run commands

imaging/service.yaml

...
commands:
  build: make build
  run: ./bin/imaging -debug
...

Because services can be written in any language, there is no specific build or execution format. Hence, the service execution engine needs to know how to build and run a service.

• The build command should be specified as a bash command that will be executed in the root folder of the service. This command can be omitted if you are not using a compiled language.
• The run command is required and should be specified as a binary or script file to execute along with its arguments. The file location should be relative to the service's root folder. Make sure to specify a shebang if you want to execute a script.

Shell commands

Do not specify bash/shell commands as run command. They will not work.

Input Streams

controller/service.yaml

...
inputs:
    service: imaging
    streams:
        - path 
...

Streams are the means by which services connect and communicate with each other, comparable to server-client connections. Though unlike many client-server connections you know, one stream only allows for simplex one-to-many communication, which allows for very efficient data transfers due to the pub-sub pattern that is utilized. Because of the simplex (= one-way) property, streams can be either read streams or write streams.

Take our imaging service for example. It exposes an output (stream) called path. From the imaging service's point of view, this is a write stream: the imaging service will write its track edge data to this stream.

Suppose that another service, called "controller" depends on this track edge data, it then specifies the imaging service and its stream as its input, this is like saying "I am the controller service and from the imaging service I want to read from the path stream" to the service execution engine. To the controller service, the path stream hence is a read stream.

One service can depend on many streams from many other services, even with cyclic dependencies. The service execution engine will confirm that all inputs can be matched with a valid output stream.

The input field in the service.yaml file is an array of objects that specify a service by name and its streams as exposed by this service. If a service does not depend on any inputs, you should specify inputs as an empty array (i.e. inputs: []).

Output Streams

imaging/service.yaml

...
outputs:
  - path
...

The outputs field is an array of strings, where each string describes the name of a stream that the service exposes. This name is thought of by the service author and should best describe the type of data that is outputted. If a service does not expose streams to other services, you should specify an empty array (i.e. outputs: []).

Configuration Options

controller/service.yaml

...
configuration:
  - name: img-width
    type: number
    tunable: false
    value: 640
  - name: img-height
    type: number
    tunable: false
    value: 480
  - name: img-fps
    type: number
    value: 30

Using the ASE software framework, you can specify (runtime) configuration options dynamically through the service.yaml file. The configuration values can be accessed in your code and can even be changed during runtime (more on that later). In short: you can modify service behavior by just editing one file, without the need to rebuild or even restart your service.

The configuration field specifies an array of "configuration option objects". For each object, you should specify:

• The name of the option property
• The type of the option. This can be either string or number. The number type will be represented as the most precise float available in your language of choice.
• The value of the option, corresponding to the chosen type
• Whether or not the option is tunable. That is, if it can be tuned during runtime. Marking an option tunable is optional. By default, options are not marked not tunable during runtime

If a service exposes no configuration options, you should specify the configuration field as an empty array (i.e. configuration: []).

Formal specification

If you are interested in the formal service.yaml specification, you can find it on Github here.

Combining Services Into a Pipeline

A service in itself is nice, but not so powerful. Only by combining multiple services you can solve more complex tasks in a specialized manner. Throughout our docs and software, we will call a collection of services a processing pipeline, or pipeline in short. This definition is a testament to our racing days, when calling our collection a pipeline made much sense: data flowed from left to right (starting from a camera frame capture, and ending in motor movement to steer the car). A pipeline example from our old documentation is shown below.

This definition made sense intuitively. Every part of the pipeline is contained in its own optimized service (each service being its own process), and it is clear in which direction data flows. Nowadays, the orchestration of services fortunately is a lot more flexible, yet simpler. Services can communicate in any pattern they want (one-to-one, many-to-many, many-to-one, one-to-many) and communication pipelines can even have cycles in their communication graphs - although there is no need to dive into any further graph theory.

Pipeline Execution

A pipeline is the smallest entity that can be executed using the ASE software framework. The pipeline execution daemon engine (called roverd) will execute a pipeline and all its services as follows:

1. It checks if all service.yaml definitions are valid in isolation: this is mere YAML validation according to the given spec
2. It checks if all service.yaml definitions are valid in conjunction with each other
   • E.g. if service A depends on stream path from service B, it will verify that (1) service B exists and is enabled in this pipeline and (2) that service B exposes an output stream called path
3. It executes the service according to its commands.run definition. It keeps track of all executed services. If one service fails, or if it receives a pipeline stop instruction, all other services will be gracefully terminated as well

You might be surprised that one service failure terminates other services too: that would defeat the point of resiliency right? The key here is that services often have real-time data dependencies. Suppose that the service that spins the motors and servo (we call this the actuator service) sets the motor speed as soon as it reads a new command from the controller service. For some reason, the controller service crashes and the actuator still waits for a command. If no command comes in due to the controller crash, the motor speed will not be modified and the Rover will keep driving with the same steering angle and possibly dangerous speed. That is just asking to hit a wall.

To avoid the above scenario, the actuator service will receive a SIGINT which it can capture and use to gracefully brake and bring the Rover to a halt. Other services can also use this interrupt to their advantage, for example to clean up left open file descriptors. If a service crashes unexpectedly, we call this a fault.